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ABSTRACT 

This document, which draws heavily on lessons learned during 
11 years of the Southern Regional Education Board's High Schools That Work 
initiative, is intended to help state leaders review their existing system 
for assessing accountability in vocational education or plan such a system. 
The document begins with a discussion of the purpose of secondary vocational 
education. The following critical considerations for accountability are 
discussed in detail: (1) raising content standards; (2) measuring academic 

achievement against high standards; (3) emphasizing and measuring technical 
literacy; and (4) considering experiences after high school. The following 
guiding principles for aligning secondary vocational education with 
accountability efforts are explained: (1) make continuous improvement in 

academic and vocational content and instruction the primary focus; (2) set 
student achievement standards high enough for students to succeed after high 
school; (3) focus on things that matter to improving student achievement; (4) 
create a data collection and reporting process that links school and 
classroom practices with student achievement and tells local school leaders 
which practices are working; (5) make business/industry and postsecondary 
leaders partners in implementing reform; (6) conduct on-site technical 
assistance visits; (7) help schools use data and information effectively; and 
(8) improve and expand the use of technical literacy tests. (MN) 
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A Look at Accountability for 
Secondary Vocational Education 

By Gene Bottoms and Richard Makin 



SREB 



This report is intended to heip state leaders review the system they have or pi an the system they 
need for accountability in secondary vocational education. Drawing heavily on lessons learned during 
11 years of the Southern Regional Education Boards High Schools That Work initiative, the report 
addresses the question posed to HSTW staff by several state legislators : " What does accountability 
look like for secondary vocational education ?” To answer this question, it is important to consider 
the purpose of secondary vocational education. 



The Purpose of Secondary Vocational Education 

Secondary vocational education is no longer just about preparing students for entry-level 
employment or jobs requiring other than a baccalaureate or an advanced degree. The purpose 
of secondary vocational education is to prepare students for both work and further education by 
1) increasing the academic achievement of students enrolled in vocational courses, 2) ensuring 
that students are technically literate, and 3) ensuring that they can continue to learn in a career 
and in postsecondary education. 

In considering an accountability system, educators and policy-makers must pay attention to 
the role of academics in secondary vocational education and to the relationships between acade- 
mic and vocational teachers in high school. Educators and policy-makers need to consider these 
facts: 



1. Quality vocational programs raise academic achievement. Success in the workplace and 
further education is based on academic knowledge and skills. Quality secondary vocational 
courses increase students’ academic achievement. 



2. Schools should be held accountable for improving the academic and technical achievement 
of students pursuing vocational studies. Vocational students should be taught the same aca- 
demic core traditionally taught to only the “best” students. They also should participate in 
authentic learning experiences that will help them apply knowledge and skills learned in 
academic classes. One way to teach a more advanced academic core to some students is to 
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get academic and vocational teachers to plan together. 

3. A two-track curriculum is unacceptable if one track serves the “best” and the other serves 
the “rest.” The SREB advocates a high school curriculum that requires all youths to com- 
plete an upgraded academic core and in-depth study in career/technical studies, advanced 
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academic studies or a combination of both. 
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4. Improvement of high school vocational/technical studies requires whole-school involvement in changing 
what is taught, how it is taught and what is expected of students in academic and vocational classes. It 
also requires structures that 1) make parents an integral part of planning and supporting their children 
in a demanding program of study, 2) enable academic and vocational teachers to work together, and 
3) make greater use of the community for learning. 

Key Considerations for Accountability 

There are key considerations for secondary vocational education in an era of accountability, including 
four critical considerations: content standards, academic achievement, technical achievement and experiences 
after high school. 

Component #1: Raising Content Standards 

Too many high school students are allowed to take repetitive low-level academic courses, avoid mathe- 
matics and science in their senior year, enroll in vocational courses for easy elective credits, and complete 
undemanding vocational assignments. Secondary vocational education accountability must focus on an 
upgraded academic core and a career concentration, including: 

• English - four full-year college-preparatory-level courses; 

• Mathematics - three or more full-year courses, with at least two credits equal to Algebra I, geometry or 
higher-level courses; 

• Science - three or more full-year courses, with two credits in chemistry, physics, applied physics or other 
lab-based college-preparatory-level science courses; 

• Social studies - at least three credits in courses equivalent to college preparatory courses; 

® Career concentration - four credits above the academic core in vocational/technical studies, academic 
studies or a combination of both. 

Data from HSTWs ites show that students who complete challenging English, mathematics and science 
courses have significantly higher achievement than students who do not. Further, students who complete all 
components of the //JTW^-recommended curriculum — an upgraded academic core and a career concentra- 
tion — have even higher achievement in reading, mathematics and science than those who do not. 

The most successful schools in the HSTW network involve students and their parents early and often 
in planning a high school program of study aimed at specific goals for after high school. Parents at these 
schools meet with their children and a school adviser to outline a four- to six-year program of study. This 
gives students a reason for attending school and makes them think about where their high school experiences 
can take them. 

Selected indicators include 1) the percentage of students who have programs of study for four years 
of high school and two years beyond, 2) the percentage of seniors completing an upgraded academic core, 

3) the percentage of seniors completing either an academic or a vocational/technical concentration of related 
courses, and 4) the percentage of students completing both an upgraded academic core and either an acade- 
mic or a vocational concentration of related courses. 
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Component #2: Measuring Academic Achievement Against High Standards 

It is not enough to give career-bound students access to an upgraded academic core and a planned 
career concentration. Academic and vocational teachers must provide instruction that helps students meet 
high achievement standards in reading, mathematics and science. States should set academic goals high 
enough so that graduates will be able to pass employment exams and avoid remedial courses in postsec- 
ondary studies. 

High school students completing a career concentration must be able to meet or exceed academic 
achievement goals. One approach is the HSTW Assessment in reading, mathematics and science, based on 
the National Assessment of Educational Progress and administered by Educational Testing Service. Students 
who complete a challenging academic core and a career concentration and meet high achievement standards 
have an advantage over students who do not. In 1997 the SREB conducted a follow-up study of more than 
6,500 graduates who participated in the 1996 HSTW Assessment. The study compared graduates who 
earned the HSTW Award of Educational Achievement 1 with those who did not. Award recipients were! 

• More likely to enroll in further study — 83 percent, compared with 56 percent; 

• More likely to attend a four-year college — 53 percent, compared with 25 percent; 

• Less likely to need remedial courses — 15 percent, compared with 31 percent; 

• Less likely to be unemployed at some point after graduation — 17 percent, compared with 25 percent. 

More than 8,000 graduates in 1998 qualified to receive the HSTW Award of Educational Achievement. 
The award is proof that these youths took the right courses and met HSTW academic achievement goals. 

In 1996 the BellSouth Corporation invited award recipients from several states to apply for jobs in the 
telecommunications field. As a result, almost 100 recipients took BellSouth’s general qualifying exam. 
Ninety-seven percent of them passed the exam, compared with a normal passing rate of 51 percent by appli- 
cants in a pre-screened recruitment pool. As a result, BellSouth will exempt from its exams any high school 
graduate who earns the Award of Educational Achievement. 

Employment and postsecondary placement exams can be used as “reality checks” as early as the 11th 
grade. Schools should invite employers and postsecondary partners to administer these exams to students 
before they enroll in college or look for work after graduation. When students gain firsthand understanding 
of what is expected and what they will need to know, they quickly see the connection between high school 
and the future. 

The primary indicator for academic achievement is the percentage of students meeting or exceeding 
established academic-achievement goals benchmarked to postsecondary studies and employers’ hiring 
standards. 



1 To earn an Award of Educational Achievement, students completed three or four credits in a vocational concentration and two of the 
following: four credits in college-preparatory- level English; three credits in mathematics, including two in courses equivalent in content 
to courses offered in the college preparatory program; and three credits in science, Including two in courses equivalent in content to 
courses offered in the college preparatory program. They also had to score at or above the SREB s goals in reading, mathematics and sci- 
ence on the 1 996 HSTW Assessment. 



Component #3: Emphasizing and Measuring Technical Literacy 

Technical literacy is the ability to 1) apply academic knowledge and skills to a broad field of technical 
studies; 2) read, understand and communicate in the language of the technical field; 3) understand technical 
concepts and principles; and 4) use technology to complete projects in a specific career/technical field. 

The HSTW Assessment has included technical literacy from the beginning, but that component has 
not been developed or implemented because of lack of funds. However, four HSTW states are using written 
technical-literacy exams! Mississippi, North Carolina, Ohio and Oklahoma. The written exams contain 
multiple-choice items and open-ended questions. In two states, the exams include performance items with 
rubrics for rating students’ work. The exams were validated by industry committees and are updated con- 
stantly. Standards are defined and communicated in some states, but not in others. 

Schools use technical literacy exams to improve instruction, adjust course syllabi, raise expectations, and 
align the curriculum with state and national standards. Some states have eliminated ineffective programs as a 
result of the exams. A technical literacy exam can be used! 

• To inform the public about the quality of a school’s vocational program; 

• As a “high-stakes” exam for students and schools; 

• To award credit for advanced placement in a postsecondary career/technical program; 

Q To demonstrate to employers that graduates possess technical literacy skills in a career field. 

Senior projects are one way to assess technical literacy. All seniors at technical high schools in Delaware 
complete such a project. The project allows them to demonstrate their grasp of problem-solving, written and 
oral communication, and academic and technical knowledge and skills. A senior project consists of three parts! 

• A research paper that expands a student’s knowledge of a career field; 

• A product related to the career field; 

• An oral presentation to a committee of teachers and community representatives. 

Industry and trade association certification of a vocational program is an important part of quality voca- 
tional studies in high school. To receive certification, a school typically must! 

® Offer a curriculum based on industry-established standards and receive endorsements from industry 
leaders; 

© Develop laboratories that meet industry requirements; 

© Give students an opportunity to take a certification exam administered by an independent testing 
service; 

© Require training and certification standards for instructors; 

© Expose students to workplace learning experiences and options. 
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Selected indicators for technical literacy include 1) the percentage of seniors passing written technical- 
literacy exams; 2) the percentage of seniors passing performance-based technical literacy exams; 3) the num- 
ber of seniors completing industry-certified programs; 4) the number of employers waiving pre-employment 
technical-literacy testing for students who pass state technical-literacy exams; 5) the percentage of seniors 
meeting the standards of a quality senior project; and 6) the percentage of students receiving advanced- 
placement credit in an apprenticeship or a postsecondary program of study. 

Component # 4 : Experiences After High School 

Experiences after high school are the fourth key consideration for aligning secondary vocational educa- 
tion with accountability efforts. These experiences include additional training, continuing education and 
salaries earned. Some states can link high school graduates electronically to employers’ records of earnings. 
The purpose is to learn whether graduates are finding work, to determine the standard industrial classifica- 
tion (SIC) in which each graduate is working, and to measure earnings of graduates. This approach holds 
promise, particularly if employers can report wages in specific occupations. 

The Florida Education and Training Placement Information Program is an interagency data-collection 
system that obtains follow-up data on former students and program participants. Various state agencies pro- 
vide graduate or participant files from their management information systems. The files include individual 
identifiers as well as demographic, socioeconomic and programmatic data. By linking to the administrative 
records of other state and federal agencies, Florida can describe the employment, military enlistment and 
continuing education experiences of participants. 

Too many students, including those completing a vocational concentration of related courses, are enter- 
ing college without proper academic preparation. Some states calculate college remediation rates by school 
district. Few, if any, can provide data on high school graduates completing a career concentration. An elec- 
tronic connection among the databases of state departments of education and postsecondary institutions is 
essential in assessing whether students completing a career concentration are being adequately prepared aca- 
demically for postsecondary studies. 

The North Carolina Department of Public Instruction conducted a study of North Carolina’s recipients 
of the 1996 HSTW Award of Educational Achievement. The purpose of the study was to determine the 
extent to which award recipients (compared with nonrecipients) had to take remedial courses at the state’s 
community colleges. Award recipients: 

9 Were less likely to take remedial courses in mathematics or English — 18 percent, compared with 53 
percent; 

9 Were more likely to be enrolled in college at the end of the first year — 82 percent, compared with 64 
percent; 

© Had a higher grade-point average for the semester and the entire year than all 1996 graduates enrolled 
in the community college system. 

Selected indicators for experiences after high school include 1) the placement rate of graduates, 2) the 
wages and annual earnings of graduates who work, 3) the continuing education rate of graduates, and 4) the 
percentage of graduates taking remedial or developmental courses in postsecondary education. 
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Eight Guiding Principles for Aligning Vocational Education 
with Accountability Initiatives 

Guiding principles for aligning vocational education with accountability initiatives include! 

1. Make continuous improvement in academic and vocational content and instruction the primary focus. 
This can be done best by designing an accountability system that connects students’ academic and tech- 
nical achievement and future success to school and classroom experiences. 

2. Set student achievement standards high enough for students to succeed after high school. Achievement 
goals must have a degree of “stretch” that ensures success after high school. They must be high enough 
so that students can pass employers’ qualifying exams and avoid remedial courses at community and 
technical colleges. Technical literacy goals should be set high enough so that students can pass employ- 
ers’ specialty exams. Setting high content and achievement goals recognizes that every school is different 
and that the nature of improvement varies widely. Not all schools will start at the same place, and not 
all of them will reach their goals at the same time. 

3. Focus on things that matter in improving student achievement. The HSTW experience indicates that it 
would be impossible for a state to implement all of the indicators in this report immediately. Instead, 
state leaders are encouraged to focus on the percentage of students 1) completing an upgraded academic 
core; 2) completing a career concentration; 3) meeting achievement goals in reading, mathematics and 
science; and 4) meeting all three of the previous conditions. 

4. Create a data-collection and reporting process that links school and classroom practices with student 
achievement and tells local school leaders which practices are working and which are not. To look at 
how school and classroom practices are affecting student achievement, states can redesign current sys- 
tems to capture data on indicators of school and classroom practices and link the information to student 
achievement. A good accountability system will generate school reports that connect academic and tech- 
nical achievement to school and classroom practices related to: 

m What students were taught; 

M What was expected of them; 

B How they were taught; 

■ What guidance and advisement they received. 

As a result of knowing the school’s strengths and weaknesses, school leaders can design and offer pro- 
fessional development aligned with the school’s plan for changing school and classroom practices to raise stu- 
dent achievement. 

Accountability reports also should allow schools to measure how well their students are doing compared 
with students at similar schools. One way is to look at what pace-setting schools are doing to raise student 
achievement. These high-performing schools can serve as observation sites and mentor schools for lower-per- 
forming schools. 
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5. Make business/industry and postsecondary leaders partners with educators in implementing reform. 
Work with business/industry and postsecondary leaders to ensure that students who take the right 
courses and meet achievement goals are recognized and rewarded for their accomplishments. 

6. Conduct on-site technical assistance visits. Schools needing the most help should receive technical assis- 
tance visits to help them improve. Technical assistance visits are designed 1) to identify best practices, 

2) to determine and justify major challenges the school must address to increase student achievement, 

3) to identify actions the school can take to address the challenges it faces over the next three years, and 

4) to present the findings to teachers and administrators. Each school receives an oral report, a written 
report and follow-up “coaching.” 

7. Help schools use data and information effectively. Using detailed information to plan improvement is 
not a natural process. Schools need help in seeing the connection among data, what is happening at the 
school, and what should be done next. An adequately funded, ongoing professional-development pro- 
gram — flexible and responsive to school needs — should be part of accountability. The professional- 
development program should support the school s plan for improving school and classroom practices 
and student achievement. 

8. Improve and expand the use of technical literacy tests. States should develop technical literacy tests that 
emphasize open-ended questions requiring understanding and application of concepts and skills in a 
career concentration. Make technical literacy tests “high-stakes” tests for students and teachers. 



Summary 

This report presents ideas to guide state leaders in linking high school vocational education to a state 
system of accountability in ways that can improve academic and technical achievement. 



Gene Bottoms is senior vice president of the Southern Regional Education Board. Richard Makin is senior director 
for planning and management for the SREBs High Schools That Work initiative. 
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